Efficient Task Sub-Delegation for Crowdsourcing
نویسندگان
چکیده
Reputation-based approaches allow a crowdsourcing system to identify reliable workers to whom tasks can be delegated. In crowdsourcing systems that can be modeled as multi-agent trust networks consist of resource constrained trustee agents (i.e., workers), workers may need to further sub-delegate tasks to others if they determine that they cannot complete all pending tasks before the stipulated deadlines. Existing reputation-based decision-making models cannot help workers decide when and to whom to sub-delegate tasks. In this paper, we proposed a reputation aware task sub-delegation (RTS) approach to bridge this gap. By jointly considering a worker’s reputation, workload, the price of its effort and its trust relationships with others, RTS can be implemented as an intelligent agent to help workers make sub-delegation decisions in a distributed manner. The resulting task allocation maximizes social welfare through efficient utilization of the collective capacity of a crowd, and provides provable performance guarantees. Experimental comparisons with state-of-the-art approaches based on the Epinions trust network demonstrate significant advantages of RTS under high workload conditions.
منابع مشابه
Mitigating Herding in Hierarchical Crowdsourcing Networks
Hierarchical crowdsourcing networks (HCNs) provide a useful mechanism for social mobilization. However, spontaneous evolution of the complex resource allocation dynamics can lead to undesirable herding behaviours in which a small group of reputable workers are overloaded while leaving other workers idle. Existing herding control mechanisms designed for typical crowdsourcing systems are not effe...
متن کاملReputation-aware task allocation for human trustees
Compared to automated entities, human trustees have two distinct characteristics: 1) they are resource constrained (with limited time and effort to serve requests), and 2) their utility is not linearly related to income. Existing research in reputation-aware task delegation did not consider these two issues together. This limits their effectiveness in humanagent collectives such as crowdsourcin...
متن کاملPerform Three Data Mining Tasks with Crowdsourcing Process
For data mining studies, because of the complexity of doing feature selection process in tasks by hand, we need to send some of labeling to the workers with crowdsourcing activities. The process of outsourcing data mining tasks to users is often handled by software systems without enough knowledge of the age or geography of the users' residence. Uncertainty about the performance of virtual user...
متن کاملSub - delegation and Trust ( Extended
Trust mechanisms can allow an agent to identify the most trustworthy entity to which a task should be delegated. Now this entity may further delegate the task, ultimately resulting in a delegation chain representing the sub-delegation process. Such delegation chains present a problem for current trust evaluation mechanisms, as they typically which reward or penalise a single agent rather than s...
متن کاملCrowdsourcing Annotation for Machine Learning in Natural Language Processing Tasks
Human annotators are critical for creating the necessary datasets to train statistical learners, but annotation cost and limited access to qualified annotators forms a data bottleneck. In recent years, researchers have investigated overcoming this obstacle using crowdsourcing, which is the delegation of a particular task to a large group of untrained individuals rather than a select trained few...
متن کامل